System And Method For Vertical Gradient Detection In Video Processing

ABSTRACT

Methods and systems for processing video information are disclosed herein and may comprise calculating at least one vertical gradient of a plurality of adjacent pixels within a current field. A two-field difference may be calculated between a plurality of pixels within a first field and a corresponding plurality of pixels within a second field. At least one pixel may be deinterlaced within the current field based at least in part on the calculated at least one vertical gradient and the calculated two-field difference. The two-field difference may indicate an amount of motion between the plurality of pixels within the first field and the corresponding plurality of pixels within the second field. Phase shifting may be applied to at least one of the plurality of pixels within the first field and the corresponding plurality of pixels within the second field to effect in-phase alignment.

RELATED APPLICATIONS

This application is a Continuation Application of, and claims priorityto, U.S. patent application entitled “System and Method for VerticalGradient Detection in Video Processing,” filed on Nov. 10, 2005 andassigned Ser. No. 11/270,999.

This application is related to the following applications, each of whichis hereby incorporated herein by reference in its entirety for allpurposes:

-   U.S. patent application Ser. No. 11/254,450, filed Oct. 20, 2005;-   U.S. patent application Ser. No. 11/254,262, filed Oct. 20, 2005;-   U.S. patent application Ser. No. 11/272,116, filed Nov. 10, 2005;-   U.S. patent application Ser. No. 11/272,113, filed Nov. 10, 2005;-   U.S. patent application Ser. No. 11/272,112, filed Nov. 10, 2005;    and-   U.S. Provisional Patent Application Ser. No. 60/687,674, filed Jun.    6, 2005.

FIELD OF THE INVENTION

Certain embodiments of the invention relate to processing of video. Morespecifically, certain embodiments of the invention relate to a systemand method for vertical gradient detection in video processing anddeinterlacing based at least in part on the detected vertical gradient.

BACKGROUND OF THE INVENTION

During interlacing, pictures that form a video frame may be captured attwo distinct time intervals. These pictures, which may be referred to asfields and which form the video frame, comprise a plurality of orderedlines. During one of the time intervals, video content for even-numberedlines may be captured, while at a subsequent time interval, videocontent for odd-numbered lines may be captured. The even-numbered linesmay be collectively referred to as a top field, while the odd-numberedlines may be collectively referred to as a bottom field. On aninterlaced display, the even-numbered lines may be presented for displayon the even-numbered lines of a display during one time interval, whilethe odd-numbered lines may be presented for display on the odd-numberedlines of the display during a subsequent time interval.

With progressive displays, however, all of the lines of the display aredisplayed at one time interval. During deinterlacing of interlacedvideo, a deinterlacing process may generate pictures for display duringa single time interval. Deinterlacing by combining content from adjacentfields, which is known as weaving, may be suitable for regions of apicture that are characterized by little or no object motion or lightingchanges, known as inter-field motion. Displaying both the top field andbottom field at the same time interval may be problematic in cases wherethe video content comprises significant motion or significant lightingchanges. Objects that are in motion are at one position when the topfield is captured and another position when the bottom field iscaptured. If the top field and the bottom field are displayed together,a comb-like, or jagged edge affect may appear with the object. This isreferred to as a weave artifact.

Alternatively, deinterlacers may generate a picture for progressivedisplay by interpolating missing lines in a field from adjacent andsurrounding lines. This is known as spatial interpolation, or “bobbing”.While spatial interpolation avoids weave artifacts in regions with highinter-field motion, spatial interpolation loses vertical detail and mayresult in a blurry picture.

Conventional methods for deinterlacing interlaced video may produceweave artifacts, for example by incorrectly biasing deinterlacingdecisions towards weaving when spatial interpolation may be moreappropriate. Similarly, conventional deinterlacing methods may oftentimes bias deinterlacing decisions towards spatial interpolation whenweaving may be a more appropriate method for deinterlacing. Furthermore,conventional deinterlacing methods may utilize a determined amount ofweaving and spatial interpolation, or “bobbing”, which may, however,result in visible artifacts such as contouring artifacts.

Further limitations and disadvantages of conventional and traditionalapproaches will become apparent to one of skill in the art, throughcomparison of such systems with the present invention as set forth inthe remainder of the present application with reference to the drawings.

BRIEF SUMMARY OF THE INVENTION

A system and method for vertical gradient detection in video processing,substantially as shown in and/or described in connection with at leastone of the figures, as set forth more completely in the claims.

Various advantages, aspects and novel features of the present invention,as well as details of an illustrated embodiment thereof, will be morefully understood from the following description and drawings.

BRIEF DESCRIPTION OF SEVERAL VIEWS OF THE DRAWINGS

FIG. 1 is a diagram illustrating vertical movement of an object betweenadjacent fields, in accordance with an embodiment of the invention.

FIG. 2 is a diagram illustrating exemplary selection of pixel samplesfor vertical gradient detection within a current field, in accordancewith an embodiment of the invention.

FIG. 3 is a block diagram of an exemplary system for deinterlacingutilizing vertical gradient detection within a current field, inaccordance with an embodiment of the invention.

FIG. 4 is a flow diagram illustrating exemplary steps for processingvideo information, in accordance with an embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

Certain aspects of the invention may be found in a method and system forvertical gradient detection in video processing. In one embodiment ofthe invention, video information may be processed by calculating anestimate of a vertical gradient of a plurality of vertically adjacentpixels within a current field. Inter-field motion may be estimatedutilizing a measurement of differences between a plurality of fields.For example, a two-field difference may be calculated between aplurality of pixels within the current field and a correspondingplurality of in-phase pixels within an adjacent field. At least one ofthe plurality of pixels within the current field and the correspondingplurality of pixels within the adjacent field may be phase shifted toeffect in-phase alignment, prior to the calculation of the two-fielddifference. The plurality of pixels within the current field and thecorresponding plurality of in-phase pixels within the adjacent field maybe filtered, prior to the calculation of the two-field difference. Ablend control value may be generated based at least in part on thecalculated at least one vertical gradient and the calculated two-fielddifference. The generated blend control value may be limited to aparticular range, and the limiting may be achieved via a look-up table,a scaling function, and/or a clipping function. A current output samplevalue may be generated, based on the blend control value.

FIG. 1 is a diagram illustrating vertical movement of an object betweenadjacent fields, in accordance with an embodiment of the invention.Referring to FIG. 1, the rectangular object 132 may have one location ina previous field in an interlaced video signal, for example, and acorresponding image 134 of object 132 may have a different location in acurrent field in the interlaced video signal. In this regard, object 134in the current video field may correspond to object 132 in the previousfield and may be obtained as a result of vertical movement by object 132between the previous field and the current field. Area 136 may berepresentative of bad weave artifacts when vertical motion of pixels ispresent between adjacent fields of the interlaced video feed. Horizontalor near horizontal edges of a moving object, as illustrated in FIG. 1,may form distorted edges if adjacent fields are weaved. The distortededges may have the appearance of alternating light and dark horizontallines, for example.

In one embodiment of the invention, one or more measures of motion ofvideo content may be calculated and combined to generate a combinedmeasure of visible motion of pixels. The combined measure may beutilized to determine whether weaving, spatial interpolation, or amixture thereof, may be used for deinterlacing. For example, thecombined measure may comprise a calculated vertical gradient within aplurality of adjacent pixels in the current field. In addition, thecombined measure may also comprise a measure of the motion betweenfields, such as the vertical motion between the corresponding objects132 and 134 in the previous and current fields, respectively. Themeasure of motion between fields may be based on a difference betweenadjacent fields and may be calculated by utilizing a plurality ofcorresponding pixels within the adjacent fields.

FIG. 2 is a diagram illustrating exemplary selection of pixel samplesfor vertical gradient detection within a current field, in accordancewith an embodiment of the invention. Referring to FIG. 2, samples 208dd, . . . , 216 dd may be selected for calculation of a verticalgradient centered at the current output sample location 212 d. Thevertical gradient may be utilized as a measure of vertical detail, whichcan be combined with an inter-field, or two-field difference, forexample. In this regard, the vertical gradient may indicate the degreeto which horizontal or near horizontal details may exist, which mayresult in visible bad weave artifacts if the content is moving. Samples210 dd and 214 dd may be selected from the current column in the currentfield 201 d so that pixel samples 210 dd and 214 dd may be locatedimmediately above and below, respectively, to a current output samplelocation 212 d within the current field 201 d. Pixel samples 208 dd and216 dd may also be selected from the current field 201 d so that pixelsample 208 dd may be located immediately above pixel sample 210 dd, andpixel sample 216 dd may be located immediately below pixel sample 214dd. The pixels samples 208 dd, . . . , 216 dd may correspond to theplurality of pixel samples 202 a from the current field.

Similarly, pixels within the previous columns 1 and 2 and the subsequentcolumns 3 and 4 within the current field 201 d may also be selected forcalculation of the vertical gradient. For example, pixels 208 db, . . ., 216 db may be selected from the previous column 1, and pixels 208 dc,. . . , 216 dc may be selected from the previous column 2. Furthermore,pixels 208 de, . . . , 216 de may be selected from the subsequent column4, and pixels 208 df, . . . , 216 df may be selected from the subsequentcolumn 5.

Vertical gradient detection may be calculated utilizing, for example, ahigh pass filter in the vertical dimension. Furthermore, to increasesensitivity in the vertical direction and decrease sensitivity in thehorizontal direction, vertical gradient detection may also utilize a lowpass filter in the horizontal direction. For example, vertical gradientdetection may use as few as two taps vertically, or it may use moretaps. The high pass function may comprise a difference function, or itmay comprise another type of function. If vertical gradient detectionuses 4 taps vertically, the vertical taps may utilize the samecoefficient magnitude, such as 1, or they could be weighted. Thehorizontal low pass function may comprise an average function, or it maycomprise another type of function. Vertical gradient detection may alsobe used to detect presence and magnitude of a vertical gradient byutilizing an absolute value function, such as “abs( ),” for example.

In one embodiment of the invention, a vertical gradient may becalculated utilizing all 20 samples illustrated in FIG. 2. In thisregard, the vertical taps of the pixels in FIG. 2 may be weighted bycoefficients (1, 2, −2, −1), for example, and the horizontal taps may beweighted by coefficients (1, 2, 2, 2, 1), for example. The presentinvention may not be limited by the number of horizontal and verticaltaps, or by the coefficients used for weighting. A plurality of othercoefficient values may be utilized with a different number of horizontaland/or vertical taps.

Referring to FIG. 2, a weighted sum of the values of pixels 208 dd, 210dd, 214 dd and 216 dd may be calculated for purposes of determiningvertical gradient centered at the current output sample location 212 d.The weighting may be performed by multiplying each of the pixel valuesby a respective coefficient (1, 2, −2, −1). For example, the sum may be1*208 d+2*210 d−2*214 d−1*216 d. The weighted sum may be referred to assum_y_d, for example. The pixel value used for this calculation may be aluma component of each pixel, or alternatively it may be a chromacomponent of each pixel, such as Cb or Cr. A similar weighted sum foreach of a plurality of horizontally adjacent columns of pixels, such asthe pixels in columns 1, 2, 4 and 5 may be calculated. For example,pixels 208 db, . . . , 216 db may be selected from the previous column1, pixels 208 dc, . . . , 216 dc may be selected from the previouscolumn 2, pixels 208 de, . . . , 216 de may be selected from thesubsequent column 4, and pixels 208 df, . . . , 216 df may be selectedfrom the subsequent column 5.

In this regard, a weighted sum, such as sum_y_b, may be calculated forcolumn 1, similarly to sum_y_d but utilizing samples (208 db, . . . ,216 db). Similarly, weighted sum sum_y_c may be calculated utilizingsamples (208 dc, . . . , 216 dc) in column 2. Weighted sum sum_y_e maybe calculated utilizing samples (208 de, . . . , 216 de) in column 4,and weighted sum sum_y_f may be calculated utilizing samples (208 df, .. . , 216 df) in column 4. A weighted sum of these five weighted sumsmay then be calculated utilizing coefficients (1, 2, 2, 2, 1) assum_y_total=1*sum_y_b+2*sum_y_c+2*sum_y_d+2*sum_y_e+1*sum_y_f. Theresulting weighted sum sum_y_total may represent a vertical gradientcentered at the current output sample location 212 d, measuring thecomponent, for example luma, used in the foregoing calculations. Inanother embodiment of the invention, additional vertical gradient valuesmay be calculated similarly measuring other components of the pixels,for example Cb or Cr. For example, sum_Cb_total may be calculated in asimilar fashion to sum_y_total, however using the Cb component of therespective pixels in the calculations. Similarly, for examplesum_Cr_total may be calculated in a similar fashion to sum_y_total,however using the Cr component of the respective pixels in thecalculations.

In another embodiment of the invention, the absolute value ofsum_y_total may be calculated. The resulting value may be referred to asvert_grad_y, for example, and may be utilized as an estimate of thevertical luma gradient in the vicinity of the current output samplelocation. The value of vert_grad_y may be utilized to estimate thepresence of horizontal, or near-horizontal, edges in the content in thecurrent field 201 d. The value of vert_grad_y may also be utilized tocontrol the calculation of a blending control as part of ade-interlacing method or system, in accordance with an embodiment of theinvention.

Alternatively, the absolute values of the sum_Cb_total and sum_Cr_totalmay also be calculated, and the resulting values may be referred to asvert_grad_Cb and vert_grad_Cr, respectively. The values of vert_grad_Cband vert_grad_Cr may be utilized as estimates of the vertical Cbgradient and the vertical Cr gradient, respectively. A weighted sum ofvert_grad_y, vert_grad_Cb and vert_grad_Cr, for example, may becalculated. The resulting sum may be referred to as vert_grad_total, forexample. The value of ver_grad_total may be utilized to control thecalculation of a blending control, for example, as part of ade-interlacing method or system, in accordance with an embodiment of theinvention.

A vertical gradient may be indicative of horizontal or near horizontaldetails in the video content, and bad weave artifacts may result fromapplying weaving to such details if the content is moving betweenadjacent fields. A high vertical gradient value combined with anindication of motion between fields may result in utilizing spatialinterpolation as a method for deinterlacing interlaced video fields fromthe current field and at least one adjacent field. Similarly, a lowgradient value may be indicative of a lack of horizontal or nearhorizontal details such that visible bad weave artifacts are less likelyto result from motion between fields and, therefore, weaving may beselected as a method for deinterlacing interlaced video fields from thecurrent field 201 d and at least one adjacent field.

FIG. 3 is a block diagram of an exemplary system for deinterlacingutilizing vertical gradient detection within a current field, inaccordance with an embodiment of the invention Referring to FIG. 3, theexemplary system 300 may comprise a processor 302, a video processingblock 304, memory 308, and a video source 306. The video source 306 maycomprise suitable circuitry, logic, and/or code and may be adapted tocommunicate raw video stream data, for example, to the video processingblock 304. The video processing block 304 may comprise suitablecircuitry, logic, and/or code and may be adapted to process the rawvideo data received from the video source 306. For example, the videoprocessing block 304 may be adapted to deinterlace interlaced video datareceived from the video source 306 using vertical gradient detection. Inthis regard, the video processing block 304 may be implemented as aspecialized video processing chip, or alternatively as a specializedfunction within a chip that comprises other functions.

The processor 302 may comprise suitable circuitry, logic, and/or codeand may be adapted to control processing of video information by thevideo processing block 304, for example. The processor 302 may comprisea system or a host processor, or a video processor such as a videoprocessing chip or embedded video processor core. The video processormay be integrated in any device that may be utilized to generate videosignals and/or display video. The memory 308 may be adapted to store rawor processed video data, such as video data processed by the videoprocessing block 304. Furthermore, the memory 308 may be utilized tostore code that may be executed by the processor 302 in connection withvideo processing tasks performed by the video processing block 304. Forexample, the memory 308 may store code that may be utilized by theprocessor 302 and the video processing block 304 for calculating avertical gradient within a current field, and utilizing the calculatedvertical gradient during deinterlacing of interlaced video received fromthe video source 306.

The processor 302 may be adapted to calculate at least one verticalgradient within a plurality of vertically adjacent pixels within acurrent field. The plurality of pixels may be received from the videosource 306. The processor 302 may deinterlace at least one pixel withinthe current field based at least in part on the calculated at least onevertical gradient.

FIG. 4 is a flow diagram illustrating exemplary steps for processingvideo information, in accordance with an embodiment of the invention.Referring to FIGS. 2 and 4, at 402, at least one vertical gradient of aplurality of adjacent pixels within a current field 201 d may becalculated. At 404, a two-field difference may be calculated between aplurality of pixels within the current field 201 d and a correspondingplurality of in-phase pixels within an adjacent field. The adjacentfield may comprise a previous field and/or a next field. At 406, a blendcontrol value may be generated based at least in part on the calculatedat least one vertical gradient and the calculated two-field difference.At 408, the generated blend control value may be scaled and limited to aparticular range. The limiting of the blend control value may beachieved via a look-up table, a scaling function, and/or a clippingfunction. At 410, a current output sample value may be generated basedon the generated blend control value. Alternatively, in step 404 adifference between a plurality of fields may be calculated between aplurality of pixels within the current field 201 d and a correspondingplurality of in-phase pixels within at least one other field, such as apreceding field of the same polarity, for example, or a following fieldof the same polarity, for example. Alternatively, in step 404 adifference between a plurality of fields may be calculated between aplurality of pixels within a previous field and a correspondingplurality of in-phase pixels within a following field, for example.

Accordingly, aspects of the invention may be realized in hardware,software, firmware, or a combination thereof. The invention may berealized in a centralized fashion in at least one computer system or ina distributed fashion where different elements are spread across severalinterconnected computer systems. Any kind of computer system or otherapparatus adapted for carrying out the methods described herein issuited. A typical combination of hardware, software and firmware may bea general-purpose computer system with a computer program that, whenbeing loaded and executed, controls the computer system such that itcarries out the methods described herein.

One embodiment of the present invention may be implemented as a boardlevel product, as a single chip, application specific integrated circuit(ASIC), or with varying levels integrated on a single chip with otherportions of the system as separate components. The degree of integrationof the system may typically be determined primarily by speed and costconsiderations. Because of the sophisticated nature of modernprocessors, it is possible to utilize a commercially availableprocessor, which may be implemented external to an ASIC implementationof the present system. Alternatively, if the processor is available asan ASIC core or logic block, then the commercially available processormay be implemented as part of an ASIC device with various functionsimplemented as firmware.

The present invention may also be embedded in a computer programproduct, which comprises all the features enabling the implementation ofthe methods described herein, and which when loaded in a computer systemis able to carry out these methods. Computer program in the presentcontext may mean, for example, any expression, in any language, code ornotation, of a set of instructions intended to cause a system having aninformation processing capability to perform a particular functioneither directly or after either or both of the following: a) conversionto another language, code or notation; b) reproduction in a differentmaterial form. However, other meanings of computer program within theunderstanding of those skilled in the art are also contemplated by thepresent invention.

While the invention has been described with reference to certainembodiments, it will be understood by those skilled in the art thatvarious changes may be made and equivalents may be substituted withoutdeparting from the scope of the present invention. In addition, manymodifications may be made to adapt a particular situation or material tothe teachings of the present invention without departing from its scope.Therefore, it is intended that the present invention not be limited tothe particular embodiments disclosed, but that the present inventionwill include all embodiments falling within the scope of the appendedclaims.

1. A method, comprising: calculating, in at least one computing device,a vertical gradient of a plurality of adjacent pixels within a firstfield of a video frame; calculating, in the at least one computingdevice, a two-field difference between a plurality of pixels within thefirst field and a plurality of corresponding pixels within a secondfield of the video frame; and deinterlacing, in the at least onecomputing device, at least a portion of the video frame based at leastin part on the calculated vertical gradient and the calculated two-fielddifference.
 2. The method of claim 1, further comprising filtering theadjacent pixels within the first field, prior to calculating thevertical gradient.
 3. The method of claim 2, wherein filtering theadjacent pixels within the first field comprises applying a first filterin a vertical direction and a second filter in a horizontal direction.4. The method of claim 3, wherein the first filter passes a higherfrequency range than the second filter.
 5. The method of claim 3,wherein the first filter comprises a difference function.
 6. The methodof claim 3, wherein the second filter comprises an averaging function.7. The method of claim 1, wherein the calculated vertical gradient isbased at least in part on a weighted sum of a plurality of the adjacentpixels.
 8. The method of claim 1, further comprising generating a blendcontrol value based at least in part on the calculated vertical gradientand the calculated two-field difference.
 9. The method of claim 8,further comprising limiting the generated blend control value to aparticular range.
 10. A system, comprising: at least one processorconfigured to calculate a vertical gradient of a plurality of adjacentpixels within a current field of a video frame; the at least oneprocessor configured to calculate a two-field difference between aplurality of pixels within a first field of the video frame and aplurality of corresponding pixels within a second field of the videoframe; and the at least one processor configured to deinterlace at leasta portion of the video frame based at least in part on the calculatedvertical gradient and the calculated two-field difference.
 11. Thesystem of claim 10, wherein the first field is the current field. 12.The system of claim 10, wherein the at least one processor is configuredto filter the adjacent pixels within the current field, prior tocalculating the vertical gradient.
 13. The system of claim 12, whereinthe at least one processor is configured to filter the adjacent pixelsby applying a first filter in a vertical direction and a second filterin a horizontal direction.
 14. The system of claim 13, wherein the firstfilter passes a higher frequency range than the second filter.
 15. Thesystem of claim 13, wherein the first filter comprises a differencefunction.
 16. The system of claim 13, wherein the second filtercomprises an averaging function.
 17. The system of claim 10, wherein thecalculated vertical gradient is based at least in part on a weighted sumof a plurality of the adjacent pixels.
 18. The system of claim 10,wherein the at least one processor is configured to generate a blendcontrol value based at least in part on the calculated vertical gradientand the calculated two-field difference.
 19. A method, comprising:calculating, in at least one computing device, a vertical gradient of aplurality of adjacent pixels within a first field of a video frame;calculating, in the at least one computing device, a two-fielddifference between a plurality of pixels within the first field and aplurality of corresponding pixels within a second field of the videoframe; and selecting, in the at least one computing device, at least oneof a plurality of video processing operations to perform on at least aportion of the video frame based at least in part on the calculatedvertical gradient and the calculated two-field difference.
 20. Themethod of claim 19, wherein at least one of the video processingoperations comprises a deinterlacing operation.
 21. The method of claim19, wherein at least one of the video processing operations comprises aweaving deinterlacing operation.
 22. The method of claim 19, wherein atleast one of the video processing operations comprises a spatialinterpolation deinterlacing operation.